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ABSTRACT 


The  Department  of  Defense  (DoD)  has  directed  the  service’s  medical  departments 
to  operate  under  managed  care  precepts.  These  precepts  have  brought  an  increased 
emphasis  on  primary  care  and  the  primary  care  providers.  This  retrospective  study 
examined  the  referrals  initiated  by  the  family  practice  physicians  of  a  116-bed  Army 
medical  treatment  facility  (MTF).  This  study  examined  whether  after  the  family  practice 
service  reorganization  there  were  any  efficiencies  in  FP  provider  utilization  baseion  a 
decline  in  referral  rates.  This  study  also  performed  a  descriptive  analysis  of  the  actual 
referrals  generated  and  their  independent  variables  in  both  the  pre-  and  post¬ 
reorganization  data.  Two  samples  of  referrals  collected  through  the  TRICARE  Service 
center  were  examined.  One  sample,  the  pre-reorganization  group  (n  =  1 129),  consisted  of 
all  referrals  initiated  by  family  practice  physicians  during  the  months  of  August  -  October 
1996.  While  the  second  sample,  the  post-reorganization  (n  =  829),  consisted  all  referrals 
initiated  during  the  three  month  period  November  1996  -  January  1997.  The  results 
demonstrate  that  family  practice  physician  utilization  increased  based  on  a  22.6  percent 
increase  in  patient  visits  and  a  3 1 . 1  percent  decrease  in  referrals. 

Five  study  variables  were  taken  from  the  Standard  Form  513  “Consultation  sheet” 
and  further  subcategorized  into  a  total  of  23  study  variables.  The  groups  were  compared, 
variable  by  variable,  using  a  student’s  t-test  to  measure  the  magnitude  and  direction  of  any 
change.  A  total  of  12  sub-category  variables  were  statistically  significant  at  p  <  .05. 

The  study  results  indicate  that  the  family  practice  service  reorganization  had  a 
positive  effect  on  physician  utilization. 
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Primary  Care  Provider  Utilization:  A  Descriptive  Analysis  of  Family  Practice 

Referral  Rates  Before  and  After  the  Family  Practice  Service  Reorganization  at 
Reynolds  Army  Community  Hospital,  Fort  Sill,  OK 

INTRODUCTION 

Background 

Today’s  changing  health  care  environment  of  rising  costs  has  forced  all  areas  of  the 
health  care  field  to  look  for  alternative  methods  of  providing  quality  medical  care.  The 
Department  of  Defense  (DoD)  has  been  one  of  the  forerunners  in  responding  to  the 
changing  environment.  In  an  effort  to  increase  access,  control  rising  costs  and  increase 
the  quality  of  health  care  for  its  beneficiaries,  DoD  has  directed  the  service’s  medical 
departments  to  operate  under  managed  care  precepts.  The  response  to  this  directive  has 
been  the  implementation  of  TRICARE.  TRICARE  is  a  DoD  health  care  reform  initiative 
initiated  at  the  Office  of  the  Assistant  Secretary  of  Defense  for  Health  Affairs 
(OASD/HA).  It  was  created  to  ensure  the  effective  execution  of  the  military  health  care 
mission  while  assuring  increased  access  and  quality  of  care,  and  controlling  the  costs 
associated  with  health  care. 

The  challenge  to  each  medical  facility  commander  is  to  manage  their  resources  as  a 
business.  This  means  that  facilities  must  deliver  quality  care  in  an  efficient  and  effective 
manner.  Under  managed  care,  the  primary  care  manager  (PCM),  typically  a  family 
practice  physician  has  become  the  focal  point.  This  is  in  part  because  of  their  role  as  the 
gatekeeper  for  the  entry  of  beneficiaries  into  the  health  care  delivery  system.  The  PCM  is 
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actively  involved  in  the  three  base  precepts  of  managed  care,  cost,  quality  and  access  to 
health  care. 

The  cost  containment  imperatives  of  managed  care  have  focused  on  inpatient  services 
and  have  driven  the  need  to  better  manage  the  utilization  of  primary  care  providers  (PCP) 
and  the  ambulatory  patient  sector. 

One  area  of  concern  directly  related  to  PCP  utilization  is  the  number  of  referrals  and 
consultations  initiated  by  the  providers.  This  includes  referrals  and  consultations  written 
to  specialists,  sub-specialists  and  ancillary  services.  Referrals  are  defined  as  the  process  of 
one  physician  transferring  some  or  all  responsibility  for  a  patient’s  health  care  either 
temporarily  or  permanently  to  another  physician  for  the  receipt  of  actual  hands  on  care 
(Ludke  1982,  Penchanski  and  Fox  1970,  Froom,  Feinbloom  and  Rosen  1984). 
Consultations  occur  when  patient  care  is  shared  for  a  specific  diagnostic  or  therapeutic 
task,  but  the  responsibility  remains  with  the  primary  physician  (Nutting,  Franks,  and 
Clancy  1992,  Froom,  Feinbloom  and  Rosen  1984,  Penchanski  and  Fox  1970).  This  is  an 
area  of  great  significance  to  the  management  of  both  the  primary  care  providers  and  the 
referral  specialty  areas.  In  the  managed  care  sector,  the  costs  associated  with  the  specialty 
care  provider  are  often  substantially  higher  than  those  associated  with  the  primary  care 
providers.  This  is  partially  due  to  the  increased  fees  associated  with  the  intensive  nature 
of  hospital  services  (Glenn,  Lawler,  Hoerl  1987,  Nutting,  Franks,  and  Clancy  1992). 
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Conditions  which  Prompted  the  Study 

Reynolds  Army  Community  Hospital  (RACH)  is  no  stranger  to  managed  care,  having 
been  a  Catchment  Area  Management  (CAM)  demonstration  site  beginning  in  1988  and 
fully  participating  in  the  Army’s  managed  care  program,  Gateway  to  Care,  from  1992  to 
1994.  The  valuable  insight  and  experience  gained  during  these  programs  have  served  the 
organization  well  in  responding  to  the  changes  in  health  care  and  health  care  management 
brought  on  by  TRICARE. 

Under  the  TRICARE  program,  RACH  has  become  a  staff  model  health  maintenance 
organization  (HMO)  and  seeks  to  enroll  eligible  beneficiaries  into  the  facility  under  the 
TRICARE  Prime  option.  An  eligible  beneficiary  is  any  individual  who  is  authorized  to 
receive  medical  care  at  a  military  facility.  This  includes  all  Federal  Service  individuals  on 
active  duty,  their  family  members,  and  Federal  Service  retirees  and  their  families.  The 
initiatives  associated  with  the  implementation  of  TRICARE  are  expected  to  increase 
access  to  primary  care  for  TRICARE  Prime  members. 

Utilization  Management  (UM)  and  Utilization  Review  (UR)  are  two  of  these  initiatives 
which  are  playing  an  increasing  role  in  the  management  of  costs,  quality  of  care  and 
access,  especially  in  the  ambulatory  sector. 

Utilization  management  is  the  planning,  organizing  and  controlling  of  health  care 
production  in  a  cost-effective  manner  while  maintaining  high  quality  care  and  contributing 
to  the  overall  goals  of  the  institution.  The  focus  of  utilization  management  is  to  focus  on 
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providing  care  at  the  most  appropriate  level  “the  right  patient  in  the  right  place  at  the  right 
time  for  the  right  amount  of  care  at  the  right  cost”  (Sandel  1996). 

There  are  three  components  to  utilization  management:  utilization  review,  case 
management  and  discharge  planning.  There  are  three  types  of  utilization  review; 
prospective,  concurrent,  and  retrospective.  Utilization  management  is  the  primary  means 
of  monitoring  outpatient  services.  Prospective  review  is  also  know  as  pre-admission.  It  is 
a  review  of  the  case  before  any  medical  care  is  initiated.  In  the  ambulatory  arena, 
prospective  review  asks  if  outpatient  services  would  be  as  efficient  and  more  cost  effective 
than  hospitalization.  Concurrent  review  involves  managing  utilization  during  the  course  of 
hospitalization.  Retrospective  review  occurs  after  the  case  is  complete,  and  the  patient 
has  been  discharged.  This  involves  reviewing  the  case  for  improprieties  and  or  mistakes. 

There  are  two  levels  of  review.  The  first  level  is  a  screening  process,  using  approved 
criteria,  to  render  decisions  as  to  the  medical  necessity  and  appropriateness  of  the  level  of 
care.  A  second  level  review  is  conducted  to  render  medical  necessity  determinations 
based  on  the  medical  expertise  of  the  reviewer  (physician.) 

InterQual  is  the  primary  utilization  management  criteria  used  by  Foundation  Health 
Federal  Services.  They  have  been  approved  to  supplement  it  with  the  Milliman  & 
Robertson  Criteria  sets  for  use  in  the  ambulatory  (outpatient)  sector.  These  criteria  sets 
are  used  to  determine  the  appropriateness  of  care.  They  are  designed  for  review  by  a  non¬ 
physician  and  are  not  intended  as  the  final  decision  maker  for  receiving  care. 

Milliman  &  Robertson’s  Health  care  Management  Guidelines  ™  were  developed  with 
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the  intent  of  describing  resource  efficient  plans  for  care  and  treatment  which  would 
reduce  some  of  the  major  costs  of  health  care.  There  are  six  volumes  of  information 
ranging  from  Inpatient  and  Surgical  Care  vol  1,  to  Skilled  Nursing  vol  6.  Volume  three, 
Ambulatory  Care  Management,  is  used  in  evaluating  the  referrals  that  are  generated  out  of 
Family  Practice. 

The  ambulatory  care  guidelines  are  divided  into  three  major  section;  Primary  Care 
guidelines  which  define  performance  standards  for  comprehensive  primary  care,  Imaging 
guidelines,  which  present  the  clinical  indications  that  should  guide  the  level  of  technology 
applied  to  obtaining  a  diagnostic  image,  and  the  ambulatory  surgery  center  management 
guidelines.  This  is  also  known  as  the  Measur  Index™.  It  is  a  cost  analysis  of  freestanding 
ambulatory  surgery  centers  and  allows  the  definition  of  costs  required  to  operate  a  facility 
productively  and  efficiently. 

The  United  States  Army  Medical  Command  expresses  their  access  issues/standards  in 
the  form  of  a  goal.  The  access  goal  for  outpatient  care  is  expressed  in  the  TRICARE 
contract  as: 

“TRICARE  Prime  enrollees  shall  have  access  to  their  PCM  services  on 
a  same-day  basis.  Access  shall  be  available  by  telephone  or 
appointment  twenty-four  (24)  hours  per  day,  seven  (7)  days  per  week, 
to  ensure  enrollees  receive  evaluation  of  initial  illness  in  a  timely 
manner.  If  the  PCM  is  not  available,  adequate  coverage  must  be 
arranged.” 

“Participating  providers  shall  ....adhere  to  the  following  standards:  (a) 
office  wait  times  for  non-emergencies  shall  not  exceed  30  minutes;  (b) 
wait  times  for  appointments  for  well  visits  shall  not  exceed  4  weeks,  1 
week  for  routine  visits,  nor  1  day  for  acute  illness.” 
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The  National  Committee  on  Quality  Assurance  (NCQA)  tracks  utilization 
management  indicators  for  HMOs  in  the  Health  Plan  Employer  Data  and  Information  Set 
(HEDIS).  HEDIS  is  a  standardized  set  of  health-plan  performance  measurements 
(Corrigan  and  Nielsen  1993).  The  primary  metric  used  to  monitor  physician  referrals  is  to 
measure  referrals  per  100  encounters:  Total  number  of  referrals  made  by  a  PCP  for  every 
100  encounters.  An  alternate  metric  is  to  measure  the  rate  per  1000  members  per  year: 
an  annualized  referral  rate  for  every  1000  members.  To  get  the  best  picture  on  how 
physicians  are  handling  cases  it  is  best  to  monitor  both  rates  (Kongstvedt  1995). 

The  idea  is  to  decrease  the  number  of  patient  visits  per  year  allowing  for  the 
enrollment  to  increase.  It  also  shifts  the  same  day  health  care  needs  to  the  patient’s 
primary  care  provide^ 

Utilization  management  duties  were  purchased  as  part  of  the  TRICARE  contract. 
Currently  there  is  no  readily  available  historical  data  on  referral  rates  at  RACH.  Different 
parts  of  the  pertinent  information  are  collected  by  different  services.  It  was  not  until 
August  1996  that  the  Family  Practice  Service  (FPS)  became  active  in  tracking  family 
practice  physician  referrals.  The  TRICARE  contractor,  Foundation  Health  Federal 
Services,  does  have  access  to  all  of  the  historical  information  concerning  referrals  that 
were  processed  through  the  TRICARE  office.  However,  release  of  this  information  in  the 
form  of  a  comprehensive  report  addressing  the  specifics  of  the  referral  origin,  specific 
service  referred  to  or  exact  diagnosis  of  referral  is  not  in  the  TRICARE  contract.  The 
contractor  does  provide  a  quarterly  report  to  the  command  addressing  the  numbers  of 
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referrals,  pre-authorizations  and  non-availability  statements  issued  by  RACH  and  the 
network  providers. 

The  FPS  was  split  between  four  separate  family  practice  (FP)  clinics.  Each  of  these 
clinics  services  a  set  population  of  assigned  units.  Pediatrics  and  Internal  Medicine 
services  have  always  been  and  remain  separate  from  FPS,  even  though  under  managed 
care  precepts  they  are  also  considered  primary  care  providers.  The  reorganization  of  the 
FPS  included  the  consolidation  of  the  four  clinics  into  two  clinics.  The  hours  of  operation 
were  also  extended.  Currently,  sick  call  is  seen  from  0700  -  0900  Monday  -  Friday,  with 
appointments  starting  at  0900  and  running  until  1700.  Under  the  FPS  reorganization,  the 
sick-call  hours  remain  the  same,  however,  the  hours  of  operation  have  been  extended. 
Appointments  begin  at  0800  and  same  day  appointments  are  available  until  1900  each 
evening.  The  primary  reason  for  this  is  to  allow  greater  and  more  convenient  access  to  the 
active  duty  beneficiaries.  There  are  four  points  to  the  rationale  behind  the  family  practice 
reorganization.  The  first  is  to  allow  greater  family  practice  physician  utilization.  Under 
the  four  clinic  set-up,  the  organization  did  not  allow  the  family  practice  physicians  the  time 
to  perform  the  evaluation  and  procedures  they  could  perform  given  their  training.  With 
the  reorganization  and  extended  hours  it  is  hypothesized  that  the  family  practice  physicians 
should  have  the  time  to  perform  more  of  those  procedures.  The  second  point  is  that  there 
should  be  increased  access  to  the  family  practice  physicians.  As  stated  earlier,  the  evening 
hours  clinic  times  were  intended  to  increase  access  for  the  active  duty  soldiers.  The  third 
point  is  that  there  should  be  an  increase  in  the  continuity  of  care.  Impaneled  beneficiaries 
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are  assigned  to  a  primary  care  team  within  their  assigned  family  practice  clinic.  The  intent 
is  for  the  beneficiaries  to  be  seen  and  treated  by  their  assigned  team.  This  allows  for  a 
more  familiar  and  personal  interaction  between  the  patient  and  health  care  team.  The 
fourth  point  is  that  there  should  be  increased  patient  satisfaction.  The  previous  three 
points  are  all  aimed  at  providing  more  accessible  and  convenient  health  care  services  while 
maintaining  the  highest  quality  of  care. 

Statement  of  the  Problem  or  Question 

The  staff  at  Reynolds  Army  Community  Hospital  (RACH)  is  attempting  to  respond  to 
the  Assistant  Secretary  of  Defense  for  Health  Affairs  (HA)  and  the  Department  of  Defense 
(DoD)  challenge  to  operate  in  a  managed  care  environment  while  coping  with  the  fiscal 
reality  of  a  limited  core  budget.  One  initiative  of  RACH  has  been  to  focus  attention  on 
utilization  management  (UM).  One  aspect  of  the  UM  monitoring  is  the  number  of 
referrals  generated  by  the  Family  Practice  Physicians  (FPP),  specifically  related  to  the  total 
numbers  going  to  specialty  services.  This  focus  is  the  basis  for  the  study  and  asks  the 
question:  Based  on  referral  rates,  will  the  reorganization  of  the  FPS  facilitate  an  increase 
in  primary  care  provider  utilization? 

Literature  Review 

There  are  numerous  current  references  in  the  literature  that  address  the  issue  of 
referral  patterns  and  rates  in  the  primary  care  arena.  While  the  sources  reviewed  herein 
cover  a  wide  spectrum  of  referral  issues,  the  focus  of  this  review  will  be  on  primary  care 
physician  referral  patterns  and  their  impact  on  the  managed  care  environment. 
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The  patient  referral  system  is  an  important  element  in  achieving  the  objectives  of 
managed  care  and  the  Primary  Care  System  (Kordy  et  al.,  1992,  Froom,  Feinbloom, 

Rosen  1984).  Patients  presenting  to  their  primary  care  providers  expect  basic  medical 
care  and  appropriate  follow-up  services.  A  patient’s  right  to  a  specialty  referral  is 
enshrined  in  the  primary  care  contract,  an  issue  that  is  rarely  debated  in  discussion  on  the 
gatekeeper  role  of  the  primary  care  physician  (Hutchinson  1993).  Thus,  patients  requiring 
further  evaluation  and  treatment  are  referred  to  a  secondary  health  care  provider.  Family 
physicians  receive  broad-based  training  and  an  important  component  of  their  domain  is  the 
diagnosis  and  management  of  frequently  occurring  illnesses  in  their  patients  within  the 
context  of  the  family  and  community  (Froom,  Feinbloom,  Rosen  1984).  Family  physicians 
provide  definitive  care  for  over  95  percent  of  patient  encounters.  The  remainder  require 
consultation,  referral  or  both  (Froom,  Feinbloom,  Rosen  1984).  Typically  a  referred 
patient  is  more  likely  to  represent  a  diagnostic  puzzle  and  have  intermediate  probabilities 
of  disease.  These  patients  are  a  subset  of  the  primary  care  practice  who  have  not 
responded  to  the  initial  interventions  or  who  require  a  more  specific  diagnostic  evaluation 
(Sox  1996).  The  patient’s  health  status  is  the  driving  force  behind  a  referral  (Elwyn  and 
Stott  1994,  Penchansky  and  Fox  1970).  An  appropriate  referral  may  lead  to  prompt 
diagnosis  and  treatment  of  conditions  that  were  beyond  the  knowledge  or  expertise  of  the 
primary  care  physician.  An  inappropriate  referral,  however,  may  lead  to  unnecessary 
testing  and  expensive,  invasive  and  risky  procedures  in  an  often  fruitless  search  for 
diagnostic  certainty  (Nutting,  Franks,  and  Clancy  1992). 
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Elwyn  and  Stott  (1994)  ask  the  question,  “What  is  appropriate  health  care?”  and 
“What  is  an  appropriate  referral?”  These  are  complex  questions  and  the  answer  will  vary 
depending  on  who  is  asked.  The  views  differ  based  largely  on  the  perspectives  of  the 
health  care  professionals,  the  patients  and  society  as  a  whole.  There  is  a  difference 
between  appropriateness  at  the  population  level,  which  is  always  constrained  by  resources, 
and  at  an  individual  level,  which  is  modified  by  the  patient’s  characteristics  and 
preferences  or  values  (Elwyn  and  Stott,  1994).  The  interaction  between  the  doctor, 
patient,  and  illness  is  part  of  the  foundation  of  primary  care  medicine. 

The  referral  of  a  primary  care  patient  to  a  specialist  is  a  common  decision  of  a 
primary  care  physician.  A  patient  referral  can  confer  major  benefits  to  patient  and 
physician.  These  benefits  include  access  to  expert  knowledge  and  experience,  and 
advanced  technical  skills  otherwise  not  available  (Froom,  Feinbloom,  Rosen  1984,  de 
Marco  et.al  ).  The  main  objective  of  a  test  referral  is  to  insure  that  the  benefits  of  a 
diagnostic  strategy  outweigh  the  potential  risks  and  have  the  greatest  net  benefits. 
However,  there  is  also  the  need  to  minimize  the  risks  of  additional  costs  and  discomfort  to 
the  patient  if  there  is  no  expectation  of  additional  benefit  from  the  referral  (Froom, 
Feinbloom,  Rosen  1984). 

There  are  two  types  of  referral;  diagnostic  referrals,  which  include  special  diagnostic 
procedures  and  emergency  evaluations  and  therapeutic  referrals,  which  are  written  to 
initiate  therapy  or  determine  a  disposition  (Cortazzo,  Guertler,  and  Rice  1993). 

Referral  rates  and  reasons  for  referral  vary  greatly  between  providers  (Zvieli  and 
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Steinherz  1992,  Elwyn  and  Stott  1994,  Froom,  Feinbloom,  Rosen  1984).  Many 
explanations  exist  for  this  variability:  differences  in  diagnostic  case  mix,  office  workload, 
access  to  specialist  services,  physicians’  low  referral  thresholds  and  increasing  rates  of 
litigation  (Zvieli  and  Steinherz  1992,  Elwyn  and  Stott  1994).  Referral  rates  are  influenced 
by  patient  factors  such  as  age,  marital  status,  parental  pressure  and  race;  physician 
variables  such  as  age  and  length  of  training  and  medical  experience;  and  community 
variables  such  as  payment  sources,  location  of  practice,  and  even  the  season  of  the  year 
(Cortazzo,  Guertler,  and  Rice  1993,  Nutting,  Franks  and  Clancy  1992,  Brock  1977, 
Penchansky  and  Fox  1970,  and  Biberman  et.al.  1996).  Lack  of  resources,  inadequate 
hospital  information  about  earlier  contacts  with  the  patient  and  poor  communication 
between  the  primary  care  physician  and  the  specialist  are  also  causes  of  referrals 
(Penchansky  and  Fox  1970,  Elwyn  and  Stott  1994,  Froom,  Feinbloom,  Rosen  1984). 
Elwyn  and  Stott  (1994),  in  their  research  on  avoidable  referrals  found  that  referral  rates 
for  primary  care  physicians  varied  widely  even  among  doctors  working  in  the  same 
environment.  Their  study  also  indicated  that  access  to  specialist  care  was  perceived  to  be 
the  major  determinant  of  referral  behavior,  the  skill  of  individual  general  practitioners  was 
the  next  most  commonly  described  influence  on  referral  behavior.  It  was  also  determined 
that  some  physicians  found  it  quicker  to  write  a  referral  letter  rather  than  look  for  missing 
information,  particularly  if  the  doctor  thought  that  the  referral  might  be  necessary  anyway. 

Shortell  and  Vahovich  (1975),  found  that  physician-related  variables  were  the  most 
important  predictors  of  referral  rates.  The  main  v  ariables  that  they  identified  were  board 
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certification,  organization  of  practice,  caseload  severity,  and  office  workload.  They  also 
determined  that  the  perceived  status  of  the  referring  physician  in  the  medical  hierarchy 
contributed  significantly  to  the  variance.  In  today’s  uncertain  world  of  primary  care,  it 
may  be  difficult  for  physicians  to  acknowledge  their  anxieties  about  decision  making.  It 
may  well  be  in  their  patients’  best  interests  for  doctors  to  use  the  “when  in  doubt,  refer” 
motto  (Hutchinson  1993). 

Patient  demand  for  a  referral  was  found  to  be  most  common  in  affluent  areas,  where 
privately  insured  patients  were  more  likely  to  request  specialty  care,  and  in  economically 
deprived  areas,  where  patient  were  more  likely  to  demand  a  second  opinion  (Elwyn  and 
Stott  1994). 

Fertig  et  al,  1993,  in  their  study  found  that  there  was  a  wide  and  unexplained 
variation  in  primary  care  physicians  referral  rates  to  hospitals.  They  found  that  the 
variation  is  often  interpreted  to  mean  that  resources  are  being  used  inefficiently  and  many 
referrals  are  unnecessary. 

In  a  closed  health  care  system,  such  as  military  medicine,  the  ability  to  identify  the 
frequency,  urgency  and  destination  of  referrals  and  consultations  can  affect  resource 
allocation  as  facilities  seek  to  meet  patient  care  demands  (Cortazzo,  Guertler,  and  Rice 
1993,  Sox  1996).  Hospitals  can  use  this  data  to  estimate  hospital  service  capabilities,  such 
as  special  procedure  and  equipment  availability,  on-call  rosters,  number  and  type  of  beds 
and  capability  to  see  referred  patients  in  a  timely  manner  (Cortazzo,  Guertler,  and  Rice 
1993). 
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In  a  study  of  the  Israeli  military  medical  system  and  the  diagnostic  characteristics  of 
patients  referred  to  a  secondary  military  medical  facility  of  the  Israel  Defense  Forces- 
Medical  Corps,  Zviety  and  Steinherz  (1992)  found  that  many  of  the  diagnoses  were  simple 
and  straightforward  in  nature  and  could  easily  be  resolved  by  the  primary  care  military 
physicians.  It  was  theorized  that  the  referring  doctors  were  uncertain  of  their  diagnoses, 
or  at  least  uncertain  about  aspects  of  their  investigation  and  management. 

It  is  probable  that  these  findings  may  prove  unique  to  a  closed  health  care  system. 

It  is  speculated  that  military  physicians  probably  refer  many  patients  simply  because  they 
rarely  have  readily  accessible  continuity  of  care  or  routine  clinical  follow-up  opportunities 
(Cortazzo,  Guertler,  and  Rice  1993).  From  these  studies  it  is  evident  that  additional 
studies  are  needed  to  evaluate  the  adequacy  and  timeliness  of  referrals,  how  to  optimize 
the  process  and  to  assess  the  economics  of  primary  care  referrals. 

In  Army  medicine,  the  Standard  Form  513  (SF  513,  Consultation  Sheet)  is  used  for 
all  requests  for  consultation  or  referral.  The  urgency  of  a  referral  is  marked  on  the  SF  513 
as  today,  72  hours,  or  routine.  In  most  civilian  primary  care  services,  referral  patterns 
usually  are  not  bound  by  such  standardized  forms  and  vary  widely,  depending  on  local 
traditions  and  physician  availability  (Brock  1977,  Cortazzo,  Guertler,  and  Rice  1993). 
Under  managed  care  and  TRICARE,  military  hospitals  typically  have  few  clinics  with  open 
appointment  systems.  Typically  only  primary  care  clinics,  general  pediatric  clinics,  and 
general  gynecology  clinics  have  appointments  easily  available  without  formal  referral 
(Cortazzo,  Guertler,  and  Rice  1993). 
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Referrals  are  initiated  by  completing  the  SF  513  and  requesting  a  follow-appointment 
with  the  appropriate  clinic.  Routine  referrals  do  not  require  approval  of  the  service  or 
clinic  to  which  the  patient  is  referred.  However,  all  referrals  must  be  routed  through  and 
approved/validated  by  the  TRICARE  service  center  for  appropriateness,  following  the 
guidelines  established  by  InterQual  and  Milliman  and  Robertson.  The  TRICARE  service 
center  also  schedules  the  appointment.  The  service  center  first  attempts  to  appoint  the 
patient  to  the  RACH  specialty  clinic,  if  access  standards  can  not  be  met,  they  schedule  the 
patient  with  a  network  provider. 

The  most  important  variable  contributing  to  patient  risk  from  the  referral  process  is 
impaired  communication  between  the  primary  care  physician  and  the  consultant  or 
referred  physician  (Froom,  Feinbloom,  Rosen  1984).  The  consultant  often  receives  an 
insufficient  medical  history  from  the  referring  physician  and  the  feedback  from  the 
consultant  to  referring  physicians  is  at  times  inadequate.  The  trust  between  the  primary 
care  physician  and  the  patient  is  easily  damaged  if  referred  specialists’  plans  for  that 
patient  bypass  the  primary  care  physician.  Unnecessary  referrals  or  admissions  are  easily 
precipitated  by  such  lack  of  professional  manners  and  etiquette  (Elwyn  and  Stott  1994). 

Physicians  with  more  experience  have  been  found  to  have  higher  referral  rates.  While 
this  at  first  seems  counterintuitive,  this  trend  may  be  explained  because  an  experienced 
physician  may  perform  a  better  assessment  of  the  patient.  It  may  also  be  a  reflection  of  a 
greater  interest  or  curiosity  that  a  physician  has  developed  through  their  increased 
knowledge  and  skills  which  makes  them  more  aware  of  serious  or  rare  diseases  and  more 


14 


likely  to  refer  (Nutting,  Franks  and  Clancy  1992,  Elwyn  and  Stott  1994).  Zviety  and 
Steinherz  (1992),  found  that  physicians  with  low  self-confidence  or  those  with  dependent 
personalities  were  more  likely  to  refer  their  patients.  Biberman  et.al.,  (1996),  found  that 
professional  development  is  associated  with  an  increased  willingness  to  assume 
responsibility.  They  also  found  increased  education  made  it  more  likely  for  them  to  change 
their  referral  practices. 

De  Marco  et.al.,  (1993)  found  that  in  the  early  1990s  several  studies  were  conducted 
and  published  which  found  no  clear  relationship  between  physician  referral  rates  and  the 
standard  of  their  clinical  care.  To  date,  a  conclusive  explanation  for  the  wide  variations  in 
primary  care  referrals  remains  elusive,  even  when  controlled  for  medical  education, 
sociodemographic  features,  morbidity,  and  deprivation  indices  (Elwyn  and  Stott  1994, 
Glenn,  Lawler  and  Hoerl  1987). 

Physician  training  has  been  found  to  be  an  indicator  of  physician  referral  patterns 
(Zvieli  and  Steinherz  1992,  Nutting,  Franks  and  Clancy  1992,  and  Elwyn  and  Stott  1994). 
Skills  in  performing  complex  procedures  receive  major  emphasis  in  postgraduate  training. 
Physicians  who  acquire  these  skills  value  them  and  fear  that  they  might  atrophy  if  they  are 
not  applied  with  moderate  frequency,  thus  resulting  in  lower  referral  rates  (Zvieli  and 
Steinherz  1992).  The  emphasis  on  new  technology  and  procedures  during  the  training 
period  may  lead  to  higher  referral  rates.  This  is  because  the  emphasis  on  technology  tends 
to  blunt  considerations  of  alternative  approaches  that  involve  knowledge  of  the  natural 
history  of  disease  process  without  intervention  (Zvieli  and  Steinherz  1992). 
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Management  of  the  utilization  of  referral  physicians  is  an  area  of  great  importance.  In 
most  managed  health  care  plans,  the  costs  associated  with  non-primary  care  professional 
services  are  substantially  greater  than  the  cost  of  primary  care  services  (Konstvedt  1995). 
Consultation  and  referral  are  the  major  avenues  through  which  family  physicians  bring  to 
bear  the  considerable  capacity  of  the  health  care  system  on  the  care  of  their  patients.  A 
better  understanding  of,  and  more  effective  strategies  for,  consultation  and  referral, 
therefore,  will  have  an  important  beneficial  impact  on  the  cost  and  quality  of  care  that 
patients  receive  (Nutting,  Franks  and  Clancy  1992). 

The  wide  variation  in  rates  of  hospital  referral  among  primary  care  physicians  is 
potentially  of  clinical  and  economic  importance.  Referral  decisions  by  primary  care 
physicians  have  an  enormous  impact  on  the  cost  and  quality  of  care  that  patients  receive. 
Studies  suggest  that  for  each  dollar  generated  by  a  family  physician,  two  dollars  are 
generated  by  the  consultant  physician,  and  four  dollars  by  the  associated  hospital  (Nutting, 
Franks,  and  Clancy  1992).  The  referring  physician  controls  the  flow  of  substantial 
economic  consequences. 

Referral  of  patients  from  one  physician  to  another  is  essential  to  providing  quality 
health  care.  However,  the  care  of  an  individual  is  not  the  only  consequence.  The  primary 
care  physician’s  decision  to  refer  a  patient  is  inevitably  followed  by  other  events.  A 
referral  results  in  professional  fees  for  the  specialist  to  whom  referred,  and  additional  costs 
such  as;  specialty  office  visits,  hospitalization,  medical  or  surgical  procedures,  and 
additional  consultations.  Each  of  these  actions  have  their  own  economic  consequence  for 
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the  patient  and  the  larger  medical  care  system  (Glenn,  Lawler,  and  Hoerl  1987,  Nutting, 
Franks,  and  Clancy  1992).  For  the  system  to  remain  financially  viable,  primary  care 
physicians  must  become  involved  in  controlling  referral  rates,  the  destination  of  referrals, 
or  both  (Nutting,  Franks,  and  Clancy  1992,  Sox  1996).  Under  the  managed  care 
gatekeeper  strategy,  primary  care  physicians  are  now  placed  at  financial  risk  for  the  costs 
of  referral  and  hospitalization  (Glenn,  Lawler,  and  Hoerl  1987).  To  any  entity  with  the 
potential  to  alter  where  and  in  what  volume  physicians’  referrals  are  made,  control  over 
referrals  has  strategic  business  implications  as  well  (Nutting,  Franks,  and  Clancy  1992). 
Every  physician  should  stay  continually  aware  of  the  extent  to  which  his  or  her  practice  is 
economically  influenced  by  physician  referred  patients  and,  particularly,  that  portion 
requiring  hospitalization  (Glenn,  Lawler,  and  Hoerl  1987). 

A  very  important  question  today  is  whether  care  by  subspecialist  physicians  is  more 
costly  than  care  by  primary  care  physicians,  and  if  so  whether  differences  in  the  spectrum 
of  patients  justify  differences  in  expenditures  (Sox  1996).  In  theory  events  initiated  by 
referral  are  easily  countable  because  they  often  correspond  with  a  charge  or  fee.  In 
practice,  assessing  the  economic  impact  of  referral  is  difficult.  In  the  current  military 
management  systems  it  is  very  hard  to  derive  the  exact  cost  of  an  individual  medical 
procedure.  General  types  of  medical  procedures  are  given  a  cost  determined  by  the 
Diagnostic  Related  Group  (DRG)  that  it  is  categorized  under. 

Glenn,  Lawler,  and  Hoerl  (1987)  found  that  general  and  family  practitioners,  general 
internists,  pediatricians  and  obstetrician-gynecologists  provide  63%  of  all  patient-physician 
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encounters  and  account  for  half  to  two-thirds  of  all  physician  referrals.  They  also  found 
that  the  cost  of  a  referral  does  not  seem  to  be  prohibitive  as  long  as  it  does  not  result  in  a 
hospital  admission  (Glenn,  Lawler,  and  Hoerl  1987). 

If  referral  rates  or  practices  are  found  to  be  inappropriate  or  to  exceed  the  utilization 
review  standards,  there  are  a  number  of  ways  in  which  to  try  to  modify  or  alter  the  current 
rates.  Two  of  the  most  discussed  in  the  literature  were  data  feedback  and  referral 
guidelines. 

Feedback  on  rates  of  referral  could  be  used  to  facilitate  practices  in  auditing  their  own 
referral  behavior  (de  Marco  et  al.,  1993,  Fertig  et  al.,  1993).  In  order  for  data  feedback  to 
be  effective,  there  must  be  a  reliable  information  system  present  in  which  the  physicians 
have  confidence.  There  are  several  different  types  of  data  sets  that  can  be  used  as  criteria 
for  data  feedback.  These  include;  details  of  referrals  from  primary  care  physicians  to 
hospitals  collected  from  hospital  computers,  referrals  to  specialty  care;  appropriateness  of 
referrals  as  judged  by  the  consultant  who  saw  the  patient,  appropriateness  of  referrals  as 
judged  by  an  independent  general  practitioner  against  a  set  of  referral  guidelines,  or  the 
effect  of  the  referral  guidelines  on  referral  rates  as  judged  by  general  practitioners  who 
used  the  guidelines  (Fertig  et  al.,  1993,  de  Marco  et  al.,  1993).  From  the  physicians  point 
of  view,  any  audit  needs  to  be  based  on  clinical  cases  that  examine  outcomes  in  terms  of 
measurable  changes  in  health  and  functions  status  rather  than  on  rates  of  referral  (de 
Marco  et  al.,  1993,  Kordy  et  al.,  1992).  In  attempting  to  implement  any  type  of  physician 
practice  modification  it  is  necessary  to  understand  that  the  perspectives  of  the  primary  care 
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physician,  the  specialist,  and  the  patient  will  probably  differ  on  appropriateness  of  the  care 
(Fertig  et  al.,  1993). 

The  most  serious  draw  back  to  data  feedback  is  that  there  is  often  wide  spread 
skepticism  among  the  concerned  physicians  about  the  accuracy  of  the  data  in  their  own 
and  other  practices  (de  Marco  et  al.,  1993,  Fertig  et  al.,  1993).  This  skepticism  produces 
very  limited  interest  in  audit  reports.  It  is  often  very  difficult  to  engage  the  doctors  in 
discussion  about  differences  in  clinical  behavior  on  the  basis  of  data  which  they  may 
regard  as  seriously  inaccurate,  de  Marco  et  al.  (1993),  found  that  doctors  from  high 
referring  practices  were  defensive  about  their  clinical  practices,  and  there  was  a  general 
feeling  that  to  be  a  low  referrer  was  a  good  thing,  if  only  because  a  low  referral  rate  was 
likely  to  protect  their  practice  against  unwelcome  attention  from  utilization  review 
committees. 

Referral  guidelines  are  criteria  sets  designed  to  assist  a  physician  in  deciding  when  it 
is  medically  appropriate  to  refer  a  patient.  The  use  of  referral  guidelines  to  establish  when 
a  referral  should  take  place,  often  fails  to  include  the  patient’s  perspective.  This  is  an 
important  consideration  because  a  patients’  expectations  of  a  referral  often  differ  from  the 
objectives  of  the  doctor  (de  Marco  et  al.,  1993).  There  is  data  to  suggest  that  strict 
application  of  the  referral  guidelines  would  have  been  unlikely  to  change  rates  of  referral 
significantly  (Fertig  et  al.,  1993). 

Although  there  are  examples  when  feedback  has  produced  alterations  in  doctors’ 
behavior,  continuing  education  and  staying  current  in  the  skills  of  their  training  are  two  of 
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the  best  mechanisms  to  help  control  primary  care  physician  referrals  today  (Elwyn  and 
Stott  1994,  Hutchinson  1993,  de  Marco  et  al.,  1993).  The  ability  of  primary  care 
physicians  to  carry  out  specific  procedures  in  practice— for  example,  sigmoidoscopy,  joint 
injection,  minor  surgeiy,  or  spinal  manipulation— was  judged  to  have  a  major  effect  in 
reducing  referrals  for  these  procedures  (Fertig  et  al.,  1993). 

Open  and  free  communication  is  also  essential  to  avoiding  unnecessary  referral  and 
modifying  inappropriate  referral  behavior.  In  addition  to  the  technical  considerations, 
appropriate  diagnostic  and  therapeutic  decisions  require  information  about  illness 
behavior,  family  factors,  patients’  expectations,  and  a  host  of  other  variables.  The  family 
physician  may  have  some  of  this  information  but  is  often  not  consulted  or  included  in  the 
decision-making  process  (Froom,  Feinbloom  and  Rosen  1984).  Many  avoidable  referrals 
that  were  due  to  limitations  of  knowledge,  attitude,  or  skills  could  have  been  pre-empted 
by  referrals  within  a  group  practice  (Elwyn  and  Stott  1994). 

Purpose 

The  purpose  of  this  study  is  two  fold:  first,  to  determine  if  utilization  of  the  family 
practice  physicians,  as  indicated  by  referral  rates,  is  increased  after  the  FPS  reorganization. 
Second,  this  study  will  conduct  a  descriptive  analysis  of  the  actual  referrals  generated  and 
their  independent  variables  in  both  the  pre-  and  post-  FPS  reorganization  data. 

The  analysis  will  focus  on  referrals  originating  from  the  family  practice  clinics  and 
directed  toward  four  specific  specialties  and  their  subspecialties,  Gynecology,  Medicine, 
Pediatrics  and  Surgery. 
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The  first  alternate  hypothesis  (Ha)  is:  utilization  of  the  primary  care  providers,  based 
on  referral  rates,  will  increase  as  a  result  of  the  primary  care  service  reorganization.  This 
will  be  seen  as  an  overall  decline  in  FP  referrals  after  the  Family  Practice  Clinics 
reorganization.  The  first  null  hypothesis  (Ho)  is:  there  will  be  no  effect  on  primary  care 
provider  utilization,  based  on  referral  rates,  after  the  FP  reorganization. 

METHOD  AND  PROCEDURES 
Changes  to  Original  Design 

In  the  process  of  working  the  issues  of  this  project,  it  was  determined  that  the  data 
set  collected  for  the  study  was  not  adequate  to  perform  the  desired  statistical  analysis. 
During  the  study,  only  the  data  from  the  actual  referrals  (SF  513)  were  collected.  It  was 
not  until  after  the  data  collection  was  completed  and  all  of  the  variables  were  coded,  that 
the  researcher  found  that  the  data  set  was  incomplete  for  the  desired  purposes. 

In  order  to  use  the  actual  referral  as  the  dependent  variable  it  would  have  been 
necessary  to  have  all  of  the  demographic  data  from  every  patient  visit  (50,148)  during  the 
study  period.  Unfortunately,  the  data  systems  in  use  did  not  have  the  ability  to  allow  the 
researcher  to  retrospectively  pull  the  required  information  from  a  data  base.  Because  of 
the  missing  data,  the  actual  referral  could  not  be  used  as  the  dependent  variable.  This  also 
prevented  any  attempt  to  analyze  the  independent  variables  for  any  potentially  strong 
associations  or  predictive  relationships  with  the  dependent  variable,  actual  referrals. 

The  original  intent  of  the  study  was  aimed  at  family  practice  physician  referral  rates 
specifically  related  to  the  independent  variables.  The  study  was  to  examine  family  practice 
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physician  referral  rates  as  a  function  of  five  independent  variables:  patient  age,  patient 
gender,  patient  beneficiary  status,  physician  experience  (#  of  years)  and  diagnosis  category 
(referral  specialty).  The  analysis  that  could  not  be  completed  was  the  determination  if  any 
of  the  variables  were  potential  predictors  of  a  referral  being  initiated. 

The  original  study  would  have  utilized  a  multiple  regression  analysis  using  stepwise 
regression.  The  purpose  of  the  regression  model  would  have  been  to  determine  which  if 
any  of  the  independant  variables  remaining  in  the  model  provided  the  best  prediction  of  a 
referral  being  generated. 

Through  consultation  with  the  researcher’s  preceptor  and  the  Chief,  FPS,  it  was 
determined  that  there  was  adequate  data  collected  to  still  perform  a  value  added  study. 

The  focus  was  shifted  from  looking  for  potential  independent  variable  predictors  of  a 
referral  to  a  descriptive  analysis  of  the  actual  referrals. 

Revised  Design 

The  population  of  this  study  is  composed  of  all  beneficiaries  eligible  to  receive  health 
care  in  a  military  facility  within  the  40  mile  radius  catchment  area  of  RACH.  There  are 
currently  28,586  enrolled  in  TRICARE  Prime.  This  population  is  composed  of  9,581 
active  duty  (AD)  soldiers,  16,226  AD  family  members,  2,749  retirees  and  their  family 
members.  There  are  also  5,218  military  students  who  are  not  eligible  to  enroll  in 
TRICARE  Prime  due  to  the  short  length  of  time  they  are  stationed  at  Fort  Sill.  They  are, 
however,  treated  as  if  they  are  enrolled.  There  are  also  1,380  Medicare  eligible  individuals 
enrolled  in  Silver  Care.  This  segment  of  the  population  has  the  highest  utilization  of  the 
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Family  Practice  Clinics.  RACH,  is  required  by  law  to  treat  all  eligible  beneficiaries.  Those 
individuals  who  have  not  enrolled  in  TRICARE  Prime  or  Silver  Care  may  make 
appointments  to  be  seen  by  the  FPS  on  a  space  available  basis  (space  A). 

The  first  phase  of  this  study  consisted  of  a  three  month  collection  of  all  referrals 
generated  from  the  Family  Practice  clinics,  prior  to  the  FPS  reorganization.  The  time 
frame  for  the  first  sampling  was  from  1  August  1996  to  31  October  1996.  This  sample  is 
the  initial  TRICARE  referral  population  and  serves  as  the  control  group.  The  second 
phase  of  the  study  consisted  of  a  three  month  collection  of  all  referrals  generated  from  the 
family  practice  clinics  following  the  FPS  reorganization.  The  time  frame  for  this  sampling 
was  from  1  November  1996  to  3 1  January  1997.  This  sample  is  the  study  group. 

The  analysis  focuses  on  the  referrals  originating  from  the  family  practice  clinics  and 
directed  to  the  four  largest  specialty  departments  at  RACH:  Obstetrics/Gynecology, 
Medicine,  Pediatrics  and  Surgery.  These  four  specialty  departments  were  chosen  as  the 
focus  of  this  study  partially  because  they  are  the  largest  specialty  departments  at  Reynolds. 
Per  discussion  with  the  Chief,  Family  Practice  Service  and  Community  Health  Services,  it 
was  clarified  that  he  wished  to  know  the  overall  referral  rates  to  these  four  services  rather 
than  the  specific  subspecialty  referral  rates  of  each  department.  He  was  more  interested  in 
what  the  overall  service  trends  were,  as  opposed  to  specific  subspecialty  rates  (Ellis  1996). 

Mayer  (1982),  in  his  study  on  referral  patterns  in  HMOs  found  that  the  four  most 
frequently  referred  to  specialties  were  general  surgery,  otolaiyngology,  orthopedics,  and 
obstetrics-gynecology.  Other  studies  site  the  specialty  departments  which  receive  the 
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largest  numbers  of  referrals  as  Medicine,  Ophthalmology,  Otolaryngology,  Surgical  and 
OB\GYN  (Kordy  et.al.,  1992,  Zvieli  1992,  Glenn,  Lawler  and  Hoerl  1987,  and  Fertig, 
et.al.,  1993).  Most  of  the  studies  included  pediatric  patient  referrals  within  one  of  the 
previous  listed  specialties.  Kordy  et.al.  (1992),  stated  that  individuals  15  years  and 
younger  represented  19.8%  of  the  overall  referral  cases  in  their  study.  For  this  study 
otolaryngology  is  included  as  part  of  medicine  while  ophthalmology  and  orthopedics  are 
combined  under  surgery.  These  four  specialty  categories  were  used  as  the  diagnosis 
category  variable  for  the  purpose  of  this  study.  A  fifth  category  labeled  “other”  was  used 
as  a  catch-all  for  the  referrals  that  did  not  fit  one  of  the  four  primary  categories.  During 
the  coding  phase,  the  variable  is  input  into  one  of  five  sub-categories,  coded  1  if  present 
and  0  if  not.  For  this  reason  the  variable  is  considered  a  mutually  exclusive,  categorically 
exhaustive  variable  (Norusis  1990  and  Motulsky  1995). 

This  study  employs  a  non-experimental  research  design  using  the  data  held  in  copies 
of  the  Standard  Form  513  (SF  513,  “Consultation  Sheet”)  to  investigate  relationships 
between  the  study  variables.  The  sole  source  of  data  for  this  study  are  the  actual  referral 
forms  (SF  513).  A  total  screening  of  all  referrals  written  during  pre-  and  post-family 
practice  reorganization  the  time  periods  was  conducted  to  collect  the  required  data. 

Age,  as  a  primary  variable,  has  been  validated  by  numerous  studies  (Kordy  et.al. 
1992,  Dale  et.  al.  1995,  Cortazzo,  Guertler,  and  Rice  1993,  Nutting,  Franks  and  Clancy 
1992,  Brock  1977,  Boling  et.al.  1992,  Glenn,  Lawler,  and  Hoerl  1987,  and  Penchansky 
and  Fox  1970).  The  main  consideration  here  is  to  determine  how  best  to  break  out  the 
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age  groups.  Textbooks  such  as  Community  Health  Analysis  by  G.E.  Alan  Dever  (1991) 
commonly  separate  a  population’s  age  break  out  in  five  year  increments.  Dale  et.  al., 
(1995)  used  an  initial  age  break  out  of  six  years  (0-5),  then  moved  to  a  ten  year  increment 
(6-16),  then  back  to  five  year  increments  until  the  age  of  sixty  (i.e.,  17-20,  21-25,  26-30, 
31-50,51-60,  >60).  Kordy  et.al.  (1992),  in  their  study  found  the  majority  of  referrals  were 
issued  to  individuals  in  the  age  group  25-44.  They  had  an  age  break  out  of:  <15,  15-24, 
25-40,  40+.  Based  on  these  results  and  previous  studies,  the  age  breakout  for  this  study 
was:  0-5,  6-15,  16-24,  25-44,  45-54,  55-64,  and  65  and  over.  It  was  felt  that  this  gave  the 
best  break  out  by  population  size.  The  over  65  population  was  of  special  interest  because 
the  facility  receives  no  funding  to  treat  these  individuals.  Because  age  was  subdivided  into 
seven  different  groupings,  it  is  considered  a  mutually  exclusive,  categorically  exhaustive 
variable.  This  is  because  during  the  coding  phase  when  the  age  variable  is  being  input  the 
subcategory  is  coded  1  if  present  and  0  if  not  (Munro  and  Page  1993). 

While  the  literature  review  did  not  reveal  any  studies  on  relationship  between  referrals 
and  military  beneficiary  status,  there  were  several  studies  on  the  relationship  between 
referrals  and  insurance  status  (Penchansky  and  Fox  1970,  Chao  et.  al.  1993,  Briggs  et.al. 
1995).  Chao  et.  al.,  (1993)  did  not  find  a  significant  relationship  between  the  disposition 
of  a  referral  and  the  insurance  status.  They  did  however,  find  that  patients  with  multiple 
consultation  requests  did  vary  significantly  from  single  consultation  requests  in  the  area  of 
insurance  status.  Patients  with  prepaid  health  plans  (20%)  or  Medicaid  or  Medicare  had  a 
statistically  significant  greater  number  of  multiple  referral  requests  than  those  with  private 
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insurance  (4%).  Penchansky  and  Fox  (1970)  found  that  pre-paid  patients  seeing 
pediatricians  were  referred  more  often  than  fee  patients.  They  also  found  that  Medicare 
and  Medicaid  patients  had  very  low  referral  rates.  The  primaiy  goal  of  using  beneficiary 
status  as  a  variable  was  to  determine  if  there  was  any  significant  relationships  between  the 
referrals  and  the  beneficiary  categories.  For  this  study,  the  four  most  commonly  seen 
beneficiary  statuses  were  used.  They  are:  active  duty  soldiers,  active  duty  family 
members,  retirees  and  their  family  members.  During  the  coding  phase,  the  variable  was 
separated  into  one  of  four  sub-categories,  coded  1  if  present  and  0  if  not.  This  then 
enabled  beneficiary  status  to  be  considered  a  mutually  exclusive,  categorically  exhaustive 
variable. 

Numerous  studies  have  cited  physician  experience  as  a  variable  related  to  referrals 
(Biberman  et.al.,  1996,  Nutting,  Franks,  and  Clancy  1992,  Brock  1977,  Boling  et.al. 
1992).  Brock  (1977)  distinguished  the  experience  differences  as  first-year  residents, 
second  year  residents  and  staff  physicians.  Boling  et.al.,  (1992)  distinguished  experience 
on  the  basis  of  being  board  certified  or  not.  None  of  the  reviewed  literature  specifically 
broke  out  experience  by  years  of  post-residency  training.  RACH  has  a  large  number  of 
family  practice  physicians  (n=24).  All  have  completed  a  family  practice  residency 
program.  There  are  both  medical  doctors  (MD)  (n=19)  and  doctors  of  osteopathy  (DO) 
(n=5).  The  experience  of  these  physicians  ranges  from  residency  graduates  with  six 
months  of  post-residency  experience  to  a  physician  with  twenty-seven  years  of  post¬ 
residency  experience.  However,  roughly  half  of  the  RACH  family  practice  physicians 
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(n=12)  have  one  year  or  less  of  post-residency  experience.  Ellis  (1996),  felt  this  lack  of 
experience  might  be  a  contributing  factor  to  potentially  higher  referrals  being  generated  by 
this  group  of  physicians.  Due  to  the  large  number  of  relatively  inexperienced  physicians 
the  experience  break  out  was:  <1  yr  (n=5),  1  yr  (n=7),  2  yrs  (n=5),  3-10  yrs  (n=4)  and 
10+yrs  (n=4).  This  gave  each  group  as  even  a  distribution  as  possible.  Physician 
experience  is  considered  a  mutually  exclusive,  categorically  exhaustive  variable.  This  was 
again  due  to  dividing  physician  experience  into  one  of  five  sub-categories  and  coding  1  if 
present  and  0  if  not. 

There  were  a  number  of  studies  in  which  gender  was  cited  as  a  potential  determining 
variable  for  a  referral  (Kordy  et.al.  1992,  Zvieli,  1992,  Chao  et.al.  1993,  Mayer  1982, 
Penchansky  and  Fox  1970).  Only  Chao  et.al.,  (1993)  and  Penchansky  and  Fox  (1970), 
specifically  stated  that  they  found  no  significant  relationship  between  gender  and  referral 
rates.  Kordy  et.al.,  (1992)  found  a  statistically  significant  difference  in  the  gender 
category  (p<0.001,  59.9%  female,  40.1%  male).  Zvieli  (1992),  found  females  were 
associated  positively  with  a  high  rate  of  patient  initiated  referrals,  (odds  ratio  3.4,  95% 
confidence  interval).  Mayer  (1982)  did  not  state  if  the  percentages  were  statistically 
significant.  Even  though  there  was  no  conclusive  data  stating  that  gender  was  a  valid 
determining  variable  of  a  referral,  it  was  felt  that  it  could  be  a  potentially  valid  statistic  for 
the  population  of  this  study.  Gender  is  a  binary  variable  and  in  this  study  males  were 
coded  1  and  female  were  coded  0. 
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The  most  commonly  cited  method  of  measuring  referral  rates  is  referrals  per  100 
patient  visits  (Brook  1977,  Bailey,  King  and  Newton  1994,  Chao  et.al.  1993,  and  Mayer 
1982).  This  study  will  use  this  measure  when  discussing  the  collective  family  practice 
physician  referral  rates.  It  will  not  attempt  to  determine  each  individual  provider’s  referral 
rate.  The  referrals  per  100  patient  visits  were  determined  by  taking  the  total  number  of 
referrals  generated  in  each  study  period  (RG)  and  dividing  by  the  total  number  of  clinic 
visits  (CV).  This  number  was  then  multiplied  by  100  to  get  a  rate  per  hundred.  The 
formula  looks  like  this:  referral  rate  (RR)  =  (RG/CV)*100. 

When  studying  systematic  relationships  between  variables  without  requiring  the  active 
control  of  those  variables,  Soeken  (1985)  recommends  non-experimental  designs.  The 
non-experimental  design  in  this  study  was  the  use  of  descriptive  statistical  analysis  to 
ascertain  any  causal  relationships  between  the  independent  variables  of  a  family  practice 
referral.  A  comparative  analysis  of  the  pre-  and  post-family  practice  reorganization 
implementation  will  determine  the  effect  that  the  implementation  may  have  had  on  the 
referrals  generated.  A  number  of  studies  have  used  correlation  analysis  in  an  attempt  to 
determine  if  there  are  any  significant  relationships  between  the  study’s  independent 
variables  and  the  dependent  variable(s)  (Biderman  et.al.  1996,  Briggs  et.al.  1995,  Chao 
et.al.  1993,  Boling  et.al.  1992,  Ludke  1982,  Zvieli  1992).  Motulsky  (1995),  recommends 
the  use  of  correlation  analysis  to  determine  which  independent  variables  may  have  a 
statistically  significant  association  with  the  dependent  variable.  In  this  case,  a  zero-order 
partial  correlation  analysis  was  used  to  ascertain  if  there  were  any  significant  relationships 
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between  independent  variables  and  the  generation  of  a  family  practice  referral.  In 
comparing  two  groups,  Motulsky  (1995),  states  the  student’s  t-test  is  an  appropriate  test 
when  groups  are  compared  variable  by  variable  for  any  differences. 

Microsoft  Excel  version  5.0  was  used  in  construction  of  the  data  sets.  No  identifying 
features  about  patients  were  used  in  the  course  of  the  study.  Names,  social  security 
numbers  and  other  potentially  identifying  patient  characteristics  were  not  incorporated  into 
the  data  sets.  Also,  by  using  retrospective  data  collection  no  direct  contact  is  necessary 
between  the  researcher  and  the  referred  patients,  thus  ensuring  patient  anonymity. 
Collectively,  these  steps  eliminated  any  potential  ethical  dilemmas  associated  with  research 
involving  patient  data  and  medical  records. 

The  concerns  of  validity,  reliability,  and  practicality  were  concerns  that  this  study 
considered  in  the  construction  and  interpretation  of  the  data  set.  By  abstracting  data  that 
was  already  recorded,  the  content  validity  of  the  Standard  Form  513  reports  were  adopted 
as  accepted  instruments.  For  those  referral/consults  with  missing  variable  data,  the 
composite  health  care  system  (CHCS)  was  accessed  to  locate  the  missing  data.  By 
serving  as  the  sole  researcher  and  using  a  strict  abstracting  process,  intra-rater  reliability 
was  ensured.  The  last  measurement  consideration  was  practicality.  Kerlinger  (1986) 
notes  that  a  tool  should  be  evaluated  in  practical  terms  of  convenience,  economy,  and 
interpretability.  The  Standard  Form  513  reports  were  readily  available,  free,  and 
understandable  (example  at  appendix  A). 
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Five  study  variables  were  taken  from  the  SF  513.  These  variables  were  subdivided  to 
further  stratify  the  sample,  yielding  a  total  of  23  variables.  The  dependent  variable  being 
the  referral  itself.  The  independent  variables  were  a  combination  of  binary  and  mutually 
exclusive,  categorically  exhaustive  variables  that  are  coded  1  if  present,  0  if  not.  The 
binary  variable  was  gender,  and  the  four  mutually  exclusive,  categorically  exhaustive 
variables  were  age,  beneficiary  status,  diagnostic  category  and  physician  experience. 

After  entering  the  raw  data  into  the  Excel  spreadsheet,  each  variable  both  binary  and 
mutually  exclusive  was  coded.  The  descriptive  statistics  were  then  calculated  for  each 
group.  In  comparing  the  two  groups,  a  student’s  t-test  was  the  statistical  measure  used  to 
determine  the  magnitude  and  direction  of  any  differences  is  the  group  means.  This  was 
accomplished  using  the  Data  Analysis  Package  of  the  Excel  v.5  software  package, 
specifically  the  “t-test:  two-sample  assuming  unequal  variance”  from  the  pull  down  menu. 
After  testing  the  mean  of  each  group,  the  inferential  statistics  were  compiled.  Next,  a 
zero-order  correlation  matrix  was  computed  (Kleinbaum,  Kupper  and  Muller  1988, 
Norusis  1990),  using  a  second  software  package,  Statistical  Package  for  Social  Sciences 
(SPSS).  The  resulting  correlation  coefficients  were  reviewed  to  determine  those  meeting 
or  exceeding  the  critical  value. 

RESULTS 

The  SF513  consult/referral  forms  originating  from  RACH  were  collected  after  they 
were  processed  through  the  TRICARE  service  center.  All  of  the  SF513s  originating  from 
the  FPS  were  separated,  counted  and  the  data  input  into  a  Microsoft  Excel  5.0 
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spreadsheet.  These  numbers  were  then  cross-checked  against  the  monthly  Family  Practice 
Service  Consult  tracking  report.  The  total  number  of  clinic  visits  and  total  number  of 
consults/referrals  generated  each  month  were  the  items  of  interest  pulled  from  this  report. 

Overall,  there  was  a  discrepancy  of  270  consults  from  what  was  reported  on  the 
monthly  Family  Practice  Service  Consult  tracking  report  and  what  was  obtained  from  the 
TRICARE  Service  Center.  In  examining  the  potential  causes  of  the  difference,  there  were 
thirty-two  consults  which  were  inadequate.  An  inadequate  referral  had  insufficient  data  to 
fully  identify  the  needed  data  fields.  This  was  typically  when  one  or  more  parts  of  the 
needed  data  were  missing  or  indecipherable.  This  data  was  needed  in  order  to  conduct  a 
search  on  CHCS.  The  other  238  missing  consult\referrals  may  be  attributable  in  part  to 
patients  not  taking  the  consult\referral  to  the  TRICARE  service  center  to  book  the 
appointment.  There  is  also  the  manner  of  the  72  hour  consult.  These  consults  are  handled 
physician  to  physician  at  RACH  due  to  the  time  constraints  imposed.  This  completely 
bypasses  the  TRICARE  Service  Center.  It  is  also  possible  that  some  of  the  consult\ 
referrals  may  have  been  “lost”  in  the  system. 

There  was  a  readily  apparent  differences  between  the  number  of  consult/referrals 
reported  by  FP  clinics  3  and  4  in  the  control  phase  and  FP  team  2  (FP  clinics  3  and  4  were 
combined  into  FP  team  2  in  the  FP  reorganization)  in  the  experimental  phase  and  the 
actual  number  of  consult/referrals  collected  through  the  TRICARE  Service  Center  for 
those  same  clinics.  It  was  in  these  clinics  that  the  largest  discrepancies  were  found. 

There  was  an  average  monthly  difference  of  45  consults/referrals  between  the  FPS  consult 
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tracking  form  and  collected  consult/  referral  totals  from  the  TRICARE  service  center. 
Overall  the  study  achieved  a  89. 17  percent  accuracy  rate  with  respect  to  the  numbers 
reported  on  the  monthly  Family  Practice  Service  Consult  tracking  report.  Statistical 
analysis  yielded  the  following  results:  mean  monthly  referrals  reported  on  FPS  Consult 
tracking  form  n  =  386,  and  mean  monthly  collected  referrals  n  =  326.  The  student’s  t-test 
reveals  that  this  difference  is  not  statistically  significant,  with  t(10)  =  0.633,  p<  0.541. 

In  comparing  the  two  data  sets,  the  first  obvious  difference  is  in  the  total  number  of 
patient  visits  to  the  FPS.  For  the  control  group,  there  were  24  FP  physicians  who  saw  a 
total  of  21,885  clinic  visits  and  generated  a  total  of  1,129  referrals.  For  the  study  group, 
there  were  24  FP  physicians  who  saw  a  total  of  28,263  clinic  visits  and  generated  a  total 
of  829  referrals.  Each  sample  represents  a  three  month  collection  period. 

As  can  be  seen  in  Table  1,  the  monthly  mean  between  the  two  groups,  7,295  pre¬ 
reorganization  and  9,421  post-reorganization,  increased  by  an  average  of  2,126  visits  per 
month,  or  +22.6  percent.  The  second  obvious  difference  is  in  the  total  number  of  referrals 
generated  by  the  FPS  during  the  study.  The  pre-reorganization  group  had  an  n  =  1,129 
while  the  post-reorganization  group  had  an  n  =  829.  As  can  be  seen  in  Table  1,  the 
monthly  mean  between  the  two  groups,  415  and  286  respectively,  decreased  by  an  average 
of  129  visits  per  month,  or  -3 1 . 1  percent.  A  comparison  of  each  group  by  variable  is 
depicted  in  Table  2.  Presented  in  this  manner,  it  allows  for  easy  contrast  of  the  two 
groups.  Differences  in  each  variable  reveal  much  about  the  workload  of  the  FPS.  Table  3 
provides  the  results  of  the  inferential  statistics  for  those  variables  found  to  be  statistically 
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significant  (p<.05)  after  all  were  tested  using  the  student’s  t-test  measure.  Variables  not 
listed  in  Table  3  were  not  statistically  significant. 

The  mean  age  for  patients  receiving  referrals  during  the  pre-reorganization  and  post¬ 
reorganization  was  31.9  and  31.3  years,  respectively.  The  mean  difference  of  0.6  years 
was  not  statistically  significant.  When  stratified  by  the  five  age  categories,  it  was 
determined  that  the  percentage  change  in  four  of  the  seven  categories  was  statistically 
significant.  The  first  age  category  was  0-5  years  old,  which  experienced  a  52.3  percent 
decrease  in  the  total  number  of  referrals  generated  and  a  2.2  percent  decrease  in  the 
overall  proportion  of  referrals.  It  was  statistically  significant  at  t  (1937)  =  2.26,  p  <  .025. 
The  second  category,  ages  25-44,  experienced  a  19.4  percent  decrease  in  the  total  number 
of  referrals  generated.  The  age  group  also  experienced  a  4.8  percent  increase  in  the 
overall  proportion  of  referrals  written  and  was  statistically  significant  at  t  (1788)  =  -2.13, 
p  <  .033.  The  third  age  group  was  the  55-64  year  group.  It  experienced  a  58.2  percent 
decrease  in  the  overall  proportion  of  referrals,  a  2. 1  percent  decrease  in  the  overall 
proportion  of  referrals  written  and  was  statistically  significant  at  t  (1953)  =  2.44,  p  <  .015. 
The  last  group  was  the  65  and  over  age  group.  It  experienced  a  53.8  percent  decrease  in 
the  total  number  of  referrals,  a  2.2  percent  decrease  in  the  overall  proportion  of  referrals 
written  and  was  statistically  significant  at  t  (1941)  =  2.25,  p  <  .024. 

When  stratified  by  beneficiary  status,  total  referrals  among  the  four  groupings 
varied  significantly  between  all  of  the  groups  except  for  the  active  duty.  Active  duty 
referrals  declined  by  45  referrals,  yet  the  overall  percentage  increased  after  the 
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TABLE  1.  AVERAGE  MONTHLY  NUMBER  OF  CONSULT/REFERRALS  FROM  THE 

FAMILY  PRACTICE  SERVICE 


Pre-Reorganization  Sample 


Month 

Sampled 

#  Consult/ 

Referrals 

Reported 

#  Consult/ 

Referrals 

Collected 

Difference 

% 

Difference 

August 

248 

203 

September 

583 

414 

October 

414 

511 

Total 

1245 

1128 

-117.00 

Monthly  Mean 

415.00 

376.00 

-39.00 

-9.4% 

StdDev 

167.50 

157.48 

Post-Reorganization  Sample 

#  Consult/ 

#  Consult/ 

Month 

Referrals 

Referrals 

Sampled 

Reported 

Collected 

November 

282 

242 

December 

296 

227 

January 

404 

360 

Total 

982 

829 

-153.00 

Monthly  Mean 

327.33 

276.33 

-51.00 

-15.6% 

Std  Dev 

66.76 

72.84 

Combined  for  Entire  Test  Period 

Total  Reported 

2227 

Monthly  Mean 

386 

Total  collected 

1957 

Monthly  Mean 

326 

-270 

-60.00  -15.5%  10  0.633  0.541 


Patient  Visits 

Pre-reorganization  Post-Reorganization 


August 

7831 

November 

8578 

September 

8213 

December 

9655 

October 

5841 

January 

10030 

Total 

21885 

Total 

28263 

Monthly  Mean 

7295 

Monthly  Mean 

9421 

2126  29.1% 

StdDev 

1273.60 

Std  Dev 

753.75 

Source:  Consult/Referrals  collected  from  the  TRICARE  Service  Center 

(SF  513s  &  FPS  Consult  Tracking  Form)  examined  during  study 
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reorganization  (58.3  percent  to  73.94  percent).  This  increase  was  statistically  significant 
t  (1887)  =  15.36,  p  <  .000.  Active  duty  family  member  referrals  dropped  by  126.  This 
was  a  45.5  percent  decrease  in  the  total  number  of  referrals,  a  6.36  percent  decrease  in  the 
overall  proportion  of  referrals  written  and  was  statistically  significant  at  t  (1880)  =  3.39,  p 
<  .000.  Total  retiree  referrals  declined  by  54.  This  62.1  percent  decrease  in  the  total 
number  of  referrals  written  and  3.02  percent  decline  in  the  overall  proportion  of  referrals 
written  was  statistically  significant  at  t  (1956)  =  3.65,  p  <  .000.  Retiree  family  members 
experienced  a  decrease  of  76  referrals  between  the  two  periods.  This  was  a  69.7  percent 
decline  in  the  total  number  of  referrals  written  and  a  5.67  percent  decrease  in  the  overall 
proportion  of  referrals.  This  was  statistically  significant  att(1937)  =  5.11,p<  .000. 

Males  received  more  referrals  than  females  in  both  periods.  Male  referrals  comprised 
64.7  percent  and  69.2  percent  for  the  pre-  and  post-  reorganization  groups.  The  4.5 
percent  increase  in  the  overall  proportion  of  male  referrals  and  subsequent  corresponding 
decrease  in  female  referrals  was  statistically  significant  at  t  (1817)  =  -2.18,  p  <  .030  for 
males  and  t  (1817)  =  2.18,  p  <  .030  for  females. 

The  mean  post-residency  experience  of  referring  physicians  during  the  pre¬ 
reorganization  and  post-reorganization  was  3.14  and  3.04  years,  respectively.  The  mean 
difference  of  0. 1  years  was  not  statistically  significant.  When  stratified  by  the  five 
experience  categories,  it  was  determined  that  the  percentage  change  in  referrals  written  in 
two  of  the  five  categories  was  statistically  significant.  Referrals  from  physicians  with  less 
than  1  year  of  post-residency  experience  declined  by  14,  yet  the  overall  percentage 
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TABLE  2.  DESCRIPTIVE  STATISTICS  FOR  DEPENDENT  AND  INDEPENDENT  VARIABLES 


Pre-FP  Reorganization  (Aug  -  Oct  96) 


Sample  n  =  1129 

%of 


Post-FP  Reorganization  (Nov  96  -  Jan  97) 

Sample  n  =  829 

%of 


Variable 

n 

Sample 

n 

Sample 

AGE 

AGE 

0-5 

70 

6.2% 

0-5 

33 

4.0% 

6-15 

43 

3.8% 

6-15 

21 

2.5% 

16-24 

265 

23.5% 

16-24 

212 

25.6% 

25-44 

562 

49.8% 

25-44 

453 

54.6% 

45-54 

69 

6.1% 

45-54 

57 

6.9% 

55-64 

55 

4.9% 

55-64 

23 

2.8% 

65+ 

65 

5.8% 

65+ 

30 

3.6% 

GENDER 

GENDER 

Male 

730 

64.7% 

Male 

574 

69.2% 

Female 

399 

35.3% 

Female 

255 

30.8% 

BENEFICIARY  STATUS 

BENEFICIARY  STATUS 

Active  Duty 

658 

58.28% 

Active  Duty 

613 

73.94% 

AD  Family  Member 

275 

24.36% 

AD  Family  Member 

150 

18.09% 

Retiree 

87 

7.71% 

Retiree 

33 

3.98% 

Retiree  Family  Member 

109 

9.65% 

Retiree  Family  Member 

33 

3.98% 

TOTAL  CLINIC  VISITS 

TOTAL  CLINIC  VISITS 

FP#1 

6229 

28.5% 

FP#1 

14483 

51.2% 

FP#2 

5153 

23.5% 

FP#2 

13780 

48.8% 

FP#3 

5528 

25.3% 

WZ3 

FP#4 

4975 

22.7% 

21IWS 

COMBINED 

(for  comparison  to  post-  reorganization  clinics) 

FP#1 

11382 

52.0% 

FP#2 

10503 

48.0% 

%  total 

clinic 

combined 

%  total 

clinic 

REFERRALS  PER  CLINIC 

referrals 

ref  rate 

clinic  rt 

REFERRALS  PER  CLINIC 

referrals 

ref  rate 

FP#1 

144 

12.8% 

2.31 

3.47 

FP#1 

338 

40.77% 

2.33 

FP#2 

251 

22.2% 

4.87 

FP#2 

491 

59.23% 

3.56 

FP#3 

262 

23.2% 

4.74 

6.99 

FP#4 

472 

41.8% 

9.49 

CLINIC  REFERRED  TO 

CLINIC  REFERRED  TO 

OB/GYN 

61 

5.40% 

OB/GYN 

46 

5.55% 

Medicine 

219 

19.40% 

Medicine 

155 

18.70% 

Pediatrics 

21 

1.86% 

Pediatrics 

11 

1.33% 

Surgery 

286 

25.42% 

Surgery 

235 

28.35% 

Other 

542 

48.10% 

Other 

382 

46.08% 

Yrs  Experience  in  FP 

Tot.# 

%  Tot. 

Yrs  Experience  in  FP 

Tot.# 

%  Tot. 

<1 

5 

20.83% 

212 

18.8% 

<1 

5 

20.83% 

198 

23.9% 

1 

6 

25.00% 

307 

27.2% 

1 

7 

29.17% 

234 

28.2% 

2 

5 

20.83% 

297 

26.3% 

2 

5 

20.83% 

186 

22.4% 

3-10 

4 

16.67% 

228 

20.2% 

3-10 

4 

16.67% 

140 

16.9% 

11+ 

4 

16.67% 

85 

7.5% 

11  + 

3 

12.50% 

71 

8.6% 

24 

24 

Family  Practice  Physician 

referral  rate  per  100 

5.16 

Family  Practice  Physician  referral  rate  per  100 

2.93 
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increased  after  the  reorganization  (18.78  percent  to  23.88  percent).  This  increase  was 
statistically  significant  at  t  (1691)  =  -2.71,  p  <  .007.  The  second  group  involved 
physicians  with  2  years  of  post-residency  experience.  Referrals  from  this  group  declined 
by  1 1 1  and  the  overall  proportion  decreased  by  3.87  percent.  This  group  just  meet  the 
criteria  for  statistical  significance  at  t  (1835)  =  1.98,  p  <  .048. 

The  percentage  of  referrals  to  each  of  the  five  different  diagnostic  categories  varied 
very  little  between  the  pre-reorganization  and  the  post-reorganization  groups.  None  of 
the  differences  were  found  statistically  significant  based  on  the  Student’s  t-test.  The 
“other”  category  was  the  largest  category  of  referrals  in  both  groups.  This  would  be 
expected  because  the  three  largest  services  categorized  under  “other”  are  Physical 
Therapy,  Occupational  Therapy  and  Nutrition/Dietary  Counseling.  These  areas  are 
Table  3.  INFERENTIAL  STATISTICS  FOR  ALL  SIGNIFICANT*  FINDINGS 


Pre-Reorganization 

Post-Reorganization 

Variable 

Mean 

Mean 

Difference 

df 

t 

n 

Active  Duty  (AD) 

0.583 

0.258 

-0.33 

1887 

15.36 

.000 

AD  Family  Member 

0.244 

0.037 

-0.21 

1880 

3.39 

.000 

Retiree  (RET) 

0.077 

0.040 

-0.04 

1956 

3.56 

.000 

RET  Family 

Member 

0.097 

0.040 

-0.06 

1937 

5.11 

.000 

Gender:  Male 

HHHESOHHI 

0.069 

0.01 

1817 

-2.18 

Gender:  Female 

| 

0.307 

-0.05 

1817 

2.18 

Age  0-5 

0.062 

0.040 

-0.02 

1937 

2.25 

Age  25-44 

0.497 

0.546 

0.05 

1788 

-2.13 

Age  55-64 

0.049 

0.028 

-0.02 

1953 

2.44 

Age  65+ 

0.058 

0.036 

-0.02 

1941 

2.25 

Experience  <1  yr 

0.187 

0.239 

0.05 

1691 

-2.71 

Experience  2  yrs 

0.263 

0.224 

-0.04 

1835 

1.98 

*  Significant  at  the  p<05  level  (two-tail) 
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considered  ancillary  services.  While  the  family  practice  physicians  are  quite  capable  of 
dealing  with  these  types  of  patients,  much  of  these  services  treatments  consist  of  therapy, 
rehabilitation  and  counseling  which  on  the  whole  is  not  proper  utilization  of  a  physician’s 
time.  Of  the  four  categories  of  interest.  Surgery  received  the  largest  number  of  referrals 
followed  by  Medicine.  This  may  be  due  to  the  more  specialized  nature  of  these  two 
categories. 

A  zero-order  correlation  analysis  was  conducted  on  both  the  pre-  and  post¬ 
reorganization  referral  samples  using  the  Statistical  Package  for  Social  Sciences 
(SPSS/PC+).  All  of  the  independent  variables  were  included  in  the  analysis  and  the 
resulting  correlation  matrixes  were  analyzed  for  any  significant  associations  between  the 
variables.  The  significant  associations  were  indicated  by  the  correlation  coefficient  and 
the  P  value.  A  correlation  coefficient  is  used  to  measure  the  direction  and  magnitude  of 
the  linear  correlation  between  variables.  The  P  value  answers  the  question:  If  there  is  no 
correlation  overall,  what  is  the  chance  that  randomly  chosen  subjects  will  correlate  as  well 
(or  better)  than  observed?  (Motulsky  1995,  Munro  and  Page  1993). 

The  purpose  of  using  the  correlation  analysis  was  to  attempt  to  identify  significant 
relationships  between  the  independent  variables.  There  is  the  possibility  that  these 
relationships  may  be  associated  with  a  higher  chance  of  a  referral  being  generated. 
However,  a  more  significant  purpose  lies  in  the  additional  questions  that  the  relationships 
may  raise,  i.e..  Why  is  a  certain  physician  experience  category  significantly  associated  with 
a  certain  age  or  diagnostic  category?,  or  Why  is  a  certain  beneficiary  category  significantly 
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associated  with  a  certain  diagnostic  category?  While  not  answerable  in  this  study,  these 
questions  raise  the  awareness  of  potential  relationships  and  present  additionaQjor  future 
studies.  It  is  also  possible  that  if  certain  independent  variables/characteristics  are  identified 
as  being  strongly  associated,  then  some  type  of  patient  preventive  health  care  educational 
process  may  be  considered  to  better  managed  the  patients  that  are  included  within  that 
particular  variable.  It  may  also  be  possible  to  identify  potential  educational  needs  of  the 
family  practice  physicians.  If  there  are  more  referrals  from  a  certain  physician  experience 
category  significantly  associated  with  a  specific  diagnostic  category,  then  there  may  be  the 
need  to  look  closer  at  the  potential  causes  for  the  higher  referral  rates. 

In  analyzing  the  correlation  matrix  of  the  independent  variables  from  both  the  pre-  and 
post-reorganization  samples,  it  was  noted  that  there  were  very  few  moderate  or  stronger 
variable  associations/correlations.  Munro  and  Page  (1993)  define  moderate  as  a 
correlation  coefficient  of  0.50-0.69.  The  complete  correlation  scaling  break-out  includes: 
little  if  any  correlation  0.00-0.25,  high  is  0.70-0.89,  and  very  high  is  0.90-1.00.  There 
were  several  correlations  which  while  they  met  the  criteria  for  significance,  were 
disregarded.  This  was  due  to  them  being  autocorrelated,  this  occurs  when  sub-categories 
of  the  same  main  grouping  were  compared  to  each  other,  i.e.,  the  0-5  year  age  grouping 
and  the  16-24  year  age  group  or  the  active  duty  and  the  active  duty  family  member 
groupings.  There  were  several  correlations  that  did  not  meet  the  correlation  significance 
of  0. 1  or  greater  yet  had  P  values  that  were  statistically  significant,  p<.05.  Motulsky 
(1995),  stated  that  this  implies  that  even  though  there  is  not  a  significant  correlation,  the 
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chance  that  randomly  chosen  subjects  would  correlate  as  well  (or  better)  than  the 
observed  subjects  was  extremely  low. 

In  analyzing  the  correlation  matrix  of  the  independent  variables  from  the  pre¬ 
reorganization,  there  were  60  correlations  that  met  or  exceeded  the  correlation 
significance  of  0. 1  or  greater.  In  the  correlation  matrix  of  the  independent  variables  from 
the  post-reorganization,  there  were  48  correlations  that  met  or  exceeded  the  correlation 
significance  of  0. 1  or  greater.  Those  associations  believed  to  be  potentially  significant  are 
shown  in  Table  4.  All  of  the  correlation  coefficients  had  a  p  value  of  .000. 

DISCUSSION 

There  are  several  interesting  results  from  this  study  that  are  worthy  of  in-depth 
elaboration.  Among  the  more  compelling  findings  are  the  increase  in  patient  visits,  overall 
decrease  in  number  of  referrals  from  the  pre-reorganization  to  the  post-reorganization,  and 
discrepancies  between  the  referral  data  reported  on  the  FPS  Consult  Tracking  Form  and 
the  SF  513s  collected  from  the  service  center.  The  results  of  the  correlation  analysis  and 
absence  of  literature  on  referral  rates  also  deserve  comment  and  discussion.  The 
limitations  of  the  study  cannot  be  overlooked  and  finishes  the  discussion  portion  of  this 
study. 

An  overall  increase  of  6,378  visits  in  the  post-reorganization  period  averages  out  to  a 
mean  monthly  increase  of  2, 126  patient  visits  per  month.  This  29. 1  percent  increase  is  not 
completely  unexpected.  A  large  part  of  the  FPS  reorganization  involved  extending  clinic 
hours  to  1900  hours  Monday  thru  Friday  and  the  closure  of  the  Minor  Care  Clinic.  The 
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Table  4.  Significant  Correlations  Pre-  and  Post-Reorganization 
variable:  pre  /  post 


Active  AD  Family  RET  Fam  Exp  Exp  Exp  Exp  Exp 

Male  Female  Duty  Mem _ Retiree  Mem _ <1  1 _ 2vrs  3-10vrs  11+vrs 


Male 

-1.00 

-1.00 

.4943 

.4738 

-.5042 

-.4404 

.1580 

-.2351 

-.2921 

.1420 

.1956 

-.1279 

-.2096 

Female  -1.00 
-1.00 

-.4943 

-.4738 

.5042 

.4404 

-.1580 

.2351 

.2921 

-.1420 

-.1956 

.1279 

.2096 

AD  .4943 

.4738 

-.4943 

-.4738 

-.6707 

-.7918 

-.3415 

-.3430 

-.3864 

-.3430 

.1308 

.2036 

.1012 

-.2903 

-.1505 

AD  Fam  -.5042 
member  -.4404 

.5042 

.4404 

-.6707 

-.7918 

-.1640 

-.1855 

-.1355 

-.1824 

.2748 

0-5 

-.1996 

-.3430 

.3247 

.4332 

.1634 

6-15 

-.2352 

-.2716 

.3183 

.3430 

16-24 

.2015 

.1779 

-.1049 

-.1194 

-.1287 

-.1194 

25-44 

.2891 

.3038 

-.1508 

-.2080 

-.1987 

-.1515 

-.1863 

-.1257 

45-54 

.1204 

.1154 

.1019 

55-64 

.1420 

-.1420 

-.2508 

-.2678 

.3037 

.3788 

65+ 

-.2690 

-.2970 

-.1225 

.3421 

.4561 

.3183 

.3900 

OB/GYN  -.3233 
-.3522 

.3233 

.3522 

-.1553 

-.2283 

.2477 

.2967 

-.1111 

Medicine 

-.1301 

.1026 

Pediatrics 

-.1361 

-.1954 

.1815 

.2467 

.1104 

Other  .1949 
.1547 

-.1949 

-.1547 

.2269 

.2014 

-.2355 

-.2082 

.1054 

.1122 

-.1919 
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appointment  templates  were  also  adjusted  to  maximize  the  number  of  patient  visits 
each  physician  could  see. 

The  impact  of  the  patient  visit  increase  was  very  significant  to  the  providers  and  staff 
who  saw  these  additional  patients.  The  FPS’s  ability  to  see  this  number  of  additional 
patients  with  the  same  size  of  staff  is  reflective  of  a  significant  increase  in  FP  physician 
utilization. 

The  significant  decrease  in  the  total  number  of  referrals  generated  by  the  FPS  during 
the  study  again  is  statistically  significant.  The  FPS  experienced  a  43.22  percent  decrease 
in  referrals  as  seen  in  Table  2.  The  mean  monthly  decrease  in  referrals  between  the  two 
groups  came  out  to  60  referrals  per  month.  There  are  two  predominate  factors  which 
contributed  to  the  decline  in  referrals.  In  August  1996,  FP  clinic  OICs  (officers  in  charge) 
were  directed  to  start  reviewing  all  specialty  referrals  for  the  appropriateness  of  referral. 
The  intent  is  for  the  OICs  to  review  each  of  the  referrals  before  they  are  sent  to  the 
TRICARE  Service  Center.  The  clinic  OICs  use  the  Milliman  &  Robertson’s  Health  care 
Management  Guidelines™.  When  they  find  an  inappropriate  referral  they  contact  the 
referring  physician  and  discuss  the  reasons  behind  the  referral.  The  discussion  is  non- 
attributional  and  is  used  as  an  educational  experience.  Foundation  Health  Federal  Services 
also  performs  an  audit  of  each  referral  as  they  receive  them.  They  also  use  the  Milliman  & 
Robertson  Criteria  sets.  They  typically  call  the  referring  physician  and  also  discuss  the 
rationale  behind  the  referral.  These  findings  are  consistent  with  the  findings  of  Benninger, 
King,  and  Nichols  (1995).  In  their  study  on  management  guidelines  for  improvement  of 
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otolaryngology  referrals  from  primary  care  physicians,  they  found  that  management  and 
referral  guidelines  were  effective  in  improving  patient  access  and  the  ratio  of  appropriate 
to  unnecessary  referrals.  Their  study  saw  a  43  percent  decrease  in  referrals  through  the 
use  of  management  guidelines. 

The  decrease  in  the  FPS  physician  referral  rate  is  statistically  significant.  The  FPS 
reduced  its  referrals  from  5.16  referrals  per  100  visits  for  the  pre-reorganization  sample 
period,  to  2.93  referrals  per  100  visits  for  the  post-reorganization  sample  period.  This 
43.2  percent  referral  rate  decrease  is  especially  noteworthy. 

No  where  in  the  literature  were  there  found  published  standards  for  physician  referral 
rates.  This  may  be  due  in  part  to  the  multitude  of  different  factors  involved  in  physician 
practices.  As  mentioned  in  the  literature  review  earlier,  there  is  a  plethora  of  variables  that 
may  impact  on  the  decision  to  refer  a  patient.  Whether  they  are  in  private  practice  or  part 
of  a  staff  model  HMO,  the  constraints  imposed  vary  as  widely  as  the  practice  set-ups. 

Numerous  reviewed  articles  mention  physician  referral  rates  varying  from  1.1  to  6.0 
referrals  per  100  office  visits  (Ludke  1982,  Chao  et.al.  1993,  Brook  1977,  Bailey  et.al. 
1994,  and  Mayer  1982).  However,  none  of  them  cited  a  nationally  accepted  or  established 
benchmark  for  physician  referrals.  The  fact  that  the  referral  rates  within  this  study  are  at 
the  lower  end  of  the  quoted  referral  rates  and  show  a  downward  trend  is  very  positive. 

We  must  accept  the  first  alternate  hypothesis  “Utilization  of  primary  care  providers, 
based  on  referral  rates,”  will  increase  as  a  result  of  the  primary  care  service  reorganization, 
it  is  evident  that  there  were  utilization  efficiencies  gained  within  the  FPS. 
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The  correlation  coefficients  produced  in  the  zero-order  correlation  matrixes  did  not 
provide  any  individually  significant  information  reference  variable  relationships.  However, 
when  combined  they  did  show  some  correlations  between  variables  that  might  possibly  be 
looked  into  for  addressing  certain  demographics  and  their  relationships  with  a  referral. 

The  results  from  the  correlation  analysis  when  looked  at  by  beneficiary  status  and  then  by 
physician  experience  revealed  significant  associations  that  may  be  related  to  a  referral 
being  generated.  The  potential  associations  are  as  follows: 

the  independent  variable  active  duty  had  significant  positive  associations  in  both  study 
samples  with  male,  age  categories  16-24  and  25-44,  physician  experience  less  than  one 
year  and  diagnostic  category  “other”.  There  was  a  significant  negative  association  with 
physician  experience  of  3-10  years. 

The  relationship  between  active  duty,  male  and  the  age  groups  16-24  and  25-44  is 
simply  a  reflection  of  the  demographics  of  active  duty  soldiers  in  the  Army.  The  questions 
raised  from  this  aspect  of  the  analysis  are:  Why  is  there  an  significant  positive  relationship 
between  the  beneficiary  status  “active  duty”  and  physicians  with  less  than  one  year  of 
experience?  and  Why  is  there  a  significant  relationship  between  active  duty  and  diagnostic 
category  “Other”.  The  second  question  may  in  part  be  related  to  the  basic  training  mission 
on  post  and  the  stresses  of  basic  training.  This  also  leads  in  to  another  question:  What 
preventive  medicine  or  educational  training  could  be  implemented  to  limit  the  number  of 
“other”  referrals? 
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The  independent  variable  active  duty  family  member  had  significant  positive 
associations  in  both  study  samples  with,  female,  age  categories  0-5  and  6-15  and 
diagnostic  categories  OB/GYN  and  pediatrics.  There  was  a  significant  negative 
association  with  physician  experience  of  less  than  one  year. 

Except  for  the  significant  negative  association  with  physician  experience  of  less  than 
one  year,  the  relationships  between  the  variables  are  merely  a  reflection  of  the 
demographics  of  active  duty  family  members.  The  question  raised  here  is:  What  is  the 
cause  of  the  negative  association  with  physicians  with  less  than  one  year  of  experience? 

The  independent  variable  retiree  had  significant  positive  associations  in  both  study 
samples  with  age  categories  45-54  and  65+.  The  independent  variable  retiree  family 
member  had  significant  positive  associations  in  both  study  samples  with  female  and  age 
category  65+.  In  both  of  the  retiree  sample  populations  there  were  no  significant 
associations  with  any  of  the  diagnostic  categories  or  physician  experience  categories.  The 
only  inferences  that  can  be  drawn  are  along  demographic  lines,  e.g.,  of  the  retirees 
receiving  referrals  are  males  between  the  age  of  45  and  65+,  retiree  family  members  are 
females  over  the  age  of  65.  This  is  again  simply  reflective  of  the  overall  population 
demographics. 

The  independent  variable,  physician  experience  less  than  one  year,  had  significant 
positive  associations  in  both  study  samples  with  active  duty  and  diagnostic  category 
“other”.  There  were  significant  negative  associations  with  female  and  active  duty  family 
member.  The  inference  here  would  be  that  the  family  practice  physicians  with  less  than 
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one  year  of  experience  refer  mostly  active  duty  males  for  diagnostic  category  “other”. 

This  may  be  attributable  to  who  pulls  sick  call  in  the  mornings. 

The  independent  variable  physician  experience  3-10  years  had  significant  negative 
association  in  both  study  samples  with  active  duty.  This  association  would  seem  to  infer 
that  for  some  reason  physicians  in  this  experience  category  do  not  see  or  refer  active  duty 
soldiers. 

Weaknesses  of  the  Study 

The  potential  for  methodology  or  construction  weaknesses  exist  in  any  study.  One 
such  weakness  in  this  study  was  the  possibility  of  seasonal  variation  affecting  the  number 
of  referrals  generated.  Typically,  it  is  felt  the  late  fall/early  winter  months  see  an  increase 
in  referrals.  Zvieli  (1992)  attempted  to  reduce  the  effect  of  seasonal  variation,  by 
conducting  his  study  on  referrals  among  military  primary  care  physicians  in  two  different 
periods  of  the  year  (Dec  85  -  Jan  86  and  Jul  -  Aug  96).  Geyman  et.  al.,  (1976),  in  their 
study  of  referrals  in  family  practice  found  a  decrease  in  referral  rates  from  February  to 
May  but  not  many  differences  in  the  other  areas.  While  this  study  is  conducted  in  two 
phases  similar  to  the  time  periods  of  the  Zvieli  study,  there  remains  the  possibility  of 
seasonal  variation.  The  potential  for  seasonal  variability  is  due  to  the  control  phase  taking 
place  during  the  summer  fall  transition,  while  the  experimental  phase  was  conducted 
during  the  fall  winter  transition.  There  is  also  an  absence  of  institutional  historical  data  for 
comparison. 
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Resource  sharing  physicians  can  only  see  CHAMPUS  eligible  beneficiaries.  This 
would  possibly  bias  the  population  that  the  active  duty  and  contract  physicians  were 
seeing.  Potentially  the  active  duty  and  contract  physicians  could  be  seeing  more  active 
duty  and  retiree  categories  of  patients  due  to  workload  guarantees  in  the  TRICARE 
contract.  Resource  sharing  physicians  were  not  included  in  this  study.  Their  absence  was 
due  to  Foundation  Federal  Health  Services  not  filling  two  provider  vacancies.  Any  fixture 
studies  may  wish  to  include  physician  relationships  with  the  organization  as  a  variable. 

The  inability  to  measure  the  number  of  visits  by  appointment  type  to  the  FPS  was  a 
second  study  weakness.  It  is  very  hard  to  make  any  valid  assumptions  about  the 
decrease/increase  in  referrals  to  a  specific  diagnostic  category,  if  the  total  number  of 
patient  visits  directly  related  to  that  category  are  not  known.  (A  decrease/increase  of  X 
referrals  may  or  may  not  be  significant  depending  on  the  actual  number  of  visits  for  that 
particular  diagnostic  category.)  The  Ambulatory  Data  System  (ADS)  will  be  able  to 
provide  significant  diagnostic  information  once  it  is  fully  implemented  and  running 
smoothly.  ADS  will  be  able  to  provide  patient  visit  numbers  as  well  as  detailed 
information  on  the  type  of  appointment  and  any  referrals  generated.  It  will  also  be  able  to 
report  this  information  by  individual  provider  as  well  as  clinic  roll-ups.  This  will  greatly 
enhance  the  ability  of  any  fixture  studies  to  provide  specific  information  on  each  type  of 
specialty  referral.  In  light  of  the  potential  for  study  weaknesses,  due  care  must  be 
exercised  when  estimating  long  term  effects  of  this  study. 
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Conclusions  and  Recommendations 


Conclusions 

This  study  had  two  primary  objectives:  First,  based  on  referral  rates,  was  family 
practice  physician  utilization  increased  after  the  FPS  reorganization?  The  second 
objective  was  to  conduct  a  descriptive  analysis  of  the  actual  referrals  generated  and  their 
independent  variables  in  both  the  pre-  and  post-  FPS  reorganization  data  and  present  the 
findings. 

Few  of  the  independent  variables  had  more  than  moderately  significant  correlations 
and  show  little  evidence  of  even  moderately  significant  relationships  with  a  referral.  The 
increases  in  patient  visits  and  decline  in  referrals  indicate  that  there  are  efficiencies  being 
gained. 

The  study  provides  conclusive  evidence  that  physician  utilization  based  on  referral 
rates  was  significantly  increased  as  a  result  of  the  Family  Practice  Service  reorganization. 
The  study  should  provide  the  organization  and  managers  of  the  FPS  with  additional 
information  concerning  the  referral  practices  within  the  FPS.  The  more  data  and 
information  that  is  collected  concerning  the  referral  practices  of  the  FPS  allows  the  FPS 
managers  to  make  better  and  well-reasoned  decisions  regarding  the  FPS.  With  the  full 
implementation  of  the  ADS  system,  more  conclusive  data  will  be  available.  Additionally, 
empirical  data  concerning  the  effect  of  the  reorganization  of  the  FPS  is  available  to  other 
MTFs  who  are  considering  restructuring  their  family  practice  services. 
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The  study  supports  the  majority  of  literature  reviewed,  indicating  that  there  are  no 
conclusive  variables  which  are  strongly  associated  with  a  referral.  The  zero-order 
correlation  matrix  results  did  provide  a  limited  demographics  profile  of  an  individual 
receiving  a  referral. 

At  this  time,  no  published  standards  for  physician  referral  rates  could  be  found  in  the 
literature  or  from  outside  sources.  There  are  various  rates  stated  in  the  literature, 
however,  none  of  them  cited  a  nationally  accepted  or  established  benchmark  for  physician 
referrals.  The  fact  that  patient  visits  increased  and  referral  rates  were  lowered  after  the 
reorganization  of  the  FPS  is  significant  and  warrants  continued  monitoring. 

Recommendations 

The  FPS  should  continue  to  monitor  and  collect  referral  data  based  on  the  five 
independent  variables  set  in  this  study.  This  will  allow  for  observations  of  trends  over 
time,  a  practice  essential  to  any  management  improvement  initiative.  Future  studies  may 
also  want  to  look  at  the  type  of  referral;  today,  72  hour,  and  routine  when  looking  at 
utilization.  The  use  of  ADS  will  allow  the  study  to  become  detailed  enough  to  look  at 
referral  rates  of  specific  diagnosises  based  on  the  CPT-9  codes  or  diagnostic  related 
groups  (DRG).  ADS  will  also  allow  the  specific  comparison  of  different  referral  variables 
based  on  total  visits  of  that  variable.  This  will  allow  more  detailed  analysis  of  referral 
rates,  trends  and  associations.  The  study  design  used  in  this  research  is  simple  enough  to 
replicate  in  any  setting  and  could  effectively  serve  as  a  template  for  on-going  analysis 
efforts.  Care  must  be  taken  to  include  all  patient  visits  in  the  data  sets. 
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The  current  initiatives  in  place  in  the  FPS  should  be  continued.  Referral  reviews  by  FP 
clinic  team  leaders  for  appropriateness  based  of  referral  guidelines  established  by  the 
organization  or  taken  from  Milliman  &  Robertson  are  effective  in  improving  both  patient 
access.  An  additional  tool  for  monitoring  FPS  referrals  would  be  to  monitor  the  ratio  of 
appropriate  to  unnecessary  referrals. 

A  prudent  review  of  the  FPS  staffing  level  and  panel  sizes  should  be  made  no  later 
than  the  one  year  anniversary  of  the  FPS  reorganization.  This  will  become  increasingly 
important  as  military  medicine  transitions  to  enrollment  based  capitation  (EBC)  as  its 
budgeting  mechanism  on  1  October  1997.  At  that  point  it  will  be  essential  to  effectively 
maximize  the  FPS  panel  sizes  and  physician  utilization. 
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